1. Field of the Invention
This invention relates, generally, to computer simulations. More specifically, it relates to providing interactive semi-automated anthropomorphic avatars that uses a human-in-the-loop paradigm to audibly and visually engage an end user (called a participant) for training and evaluation purposes.
2. Brief Description of the Related Art
Digital puppetry refers to the interactive control of virtual characters. The artistry of using a human to control the actions of animated characters in real-time has been employed for several decades in a number of venues including television and location-based entertainment. More recently, these techniques have found application in interactive networked simulations.
By affording responsive and adaptive behaviors while operating through the changeable context of an invented digital character, puppetry greatly increases the effectiveness of interactive experiences. Not unexpectedly, these benefits come with demanding requirements both from the puppeteers and from the technical aspects of projecting the simulation. From the puppeteer, there is the cognitive load associated with rapid switches between puppets; maintaining individual puppet behaviors during switches; following training scenario scripts; operating scenario specific controls; and operating puppet specific behavior controls. From a technical aspect, there is the need to reliably and efficiently capture and transmit behaviors over a long haul network; the necessity to achieve this transmission even in the presence of limited resources, including bandwidth and computational power; and the requirement to articulate these actions at the client site, providing smooth interactive performance.
One of the most prevalent and affordable examples of projected puppetry applications is the networked first person shooter game where individual players log in online to control a puppet that can do a fixed set of behaviors such as walk, run, duck, jump, punch, kick, aim and shoot using devices such as game controllers, keyboards and mice. While the process of pressing a button to control a character may not be initially intuitive, devoted players to these games have become quite adept at using these controls under the pressure of competitive game play. The control data being transmitted over long-haul worldwide networks from the buttons and dual axis joysticks found in typical controllers is insignificant compared to the incoming game state stream or the streaming VoIP that players use to collaborate during play.
The ability for players to collaborate in teams led to the emergence of machinima where remote participants could interact with each other to create short playful movies where each player acted out a predefined role, while the results were recorded through the game camera. Though the control model allowed for small efficient network streaming, with the ability to run, walk, duck, jump, aim and shoot it was fairly clear that the game characters lacked the more complex facial expressions and body language to do live acting.
In an online social networking game, such as SECOND LIFE, an array of user-playable puppet animations are provided by default with the option of players increasing this set with their own custom animations. These are used much like emoticon symbols embedded in a text document. While this is an improvement over the FPS game control by showing both facial emotion and body language, even if a puppeteer were able to master such a sizable abstract mapping, the performance still lacks the subtle hand and body motions used to communicate non-verbally and to punctuate speech.
Accordingly, what is needed is a more effective system and methodology for digital puppeteering that can capture detailed, complex facial expressions and hand and body motions. However, in view of the art considered as a whole at the time the present invention was made, it was not obvious to those of ordinary skill in the field of this invention how the shortcomings of the prior art could be overcome.
The following references are incorporated herein by reference in their entireties: Nagendran A, Pillat R, Kavanaugh A, Welch G, Hughes C E. (2013). AMITIES: Avatar-Mediated Interactive Training and Individualized Experiences System. Proceedings of 19th ACM Virtual Reality Software & Technology (VRST) 2013, Singapore, 143-152; and Nagendran A, Pillat R, Hughes C E, Welch G. (2012). Continuum of Virtual-Human Space: Towards Improved Interaction Strategies for Physical—Virtual Avatars. ACM SIGGRAPH VRCAI 2012, Dec. 2-4, 2012, Singapore, 135-142. Where a definition or use of a term in a reference, which is incorporated by reference herein, is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.
While certain aspects of conventional technologies have been discussed to facilitate disclosure of the invention, Applicants in no way disclaim these technical aspects, and it is contemplated that the claimed invention may encompass one or more of the conventional technical aspects discussed herein.
The present invention may address one or more of the problems and deficiencies of the prior art discussed above. However, it is contemplated that the invention may prove useful in addressing other problems and deficiencies in a number of technical areas. Therefore, the claimed invention should not necessarily be construed as limited to addressing any of the particular problems or deficiencies discussed herein. For instance, one can use the paradigm for remote tele-presence, where an avatar is used as a stand-in for a person operating it (referred to as a surrogate).
In this specification, where a document, act or item of knowledge is referred to or discussed, this reference or discussion is not an admission that the document, act or item of knowledge or any combination thereof was at the priority date, publicly available, known to the public, part of common general knowledge, or otherwise constitutes prior art under the applicable statutory provisions; or is known to be relevant to an attempt to solve any problem with which this specification is concerned.